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EXECUTIVE  SUMMARY 


This  report  describes  in  detail  the  procedures,  methods,  and  resources  Parsons  used  to  complete 
demonstration  projects  at  the  Former  Camp  Beale,  CA;  Former  Pole  Mountain  Target  and 
Maneuver  Area  (PMTMA),  WY ;  Fort  Sill,  OK;  Massachusetts  Military  Reservation  (MMR),  MA; 
former  Camp  George  West,  CO;  and  former  Waikoloa  Maneuver  Area  (WMA),  HI,  for 
Environmental  Security  Technology  Certification  Program  (ESTCP)  Munitions  Response  (MR) 
projects  MR-201 104  (Evaluation  and  Discrimination  Technologies  and  Classification  Results)  and 
ESTCP  MR-201157  (Demonstration  of  MetalMapper  Static  Data  Acquisition  and  Data  Analysis). 

OBJECTIVES  OF  THE  DEMONSTRATION 

To  varying  degrees,  these  projects  were  conducted  with  the  following  two  primary  objectives: 

1.  Test  and  validate  detection  and  discrimination  capabilities  of  currently  available  and 
emerging  advanced  electromagnetic  induction  (EMI)  sensors  developed  specifically  for 
discrimination  on  real  sites  under  operational  conditions. 

2.  Investigate  in  cooperation  with  regulators  and  program  managers  how  classification 
technologies  can  be  implemented  in  munitions  and  explosives  of  concern  (MEC)  cleanup 
operations. 

TECHNOLOGY  DESCRIPTION 

Two  advanced  EMI  sensors — the  MetalMapper  and  the  man-portable  Time-domain 
Electromagnetic  Multi-sensor  Towed  Array  Detection  System  (TEMTADS)  2x2  (referred  to  here 
as  TEMTADS),  were  tested  as  part  of  these  projects.  Both  sensors  employ  multiple  transmitters  to 
induce  electromagnetic  fields  in  different  orientations  or  locations  relative  to  a  given  source  object, 
and  multiple  receivers  oriented  in  three  directions  (X,  Y,  and  Z)  to  accurately  measure  the 
electromagnetic  response  generated  by  the  source  regardless  of  its  orientation  relative  to  the 
instrument.  Both  instruments  also  measure  response  over  a  significantly  longer  period  than  more 
traditional  electromagnetic  sensors.  The  MetalMapper  uses  three  orthogonally-oriented 
transmitters  to  induce  electromagnetic  fields  in  the  X,  Y,  and  Z  directions  and  seven  triaxial 
receivers  to  measure  the  response  generated  by  a  source  object  following  a  pulse  from  each 
transmitter.  The  TEMTADS  uses  four  horizontally-oriented  (Z-direction)  transmitters  arranged  in 
a  2x2  array,  and  four  triaxial  receivers — each  situated  in  the  middle  of  the  transmitters — to 
accomplish  the  same  task.  Using  the  standard  settings  applied  for  each  instrument  over  the  course 
of  these  projects,  the  MetalMapper  typically  measured  response  to  approximately  8  microseconds 
following  each  transmitter  pulse,  while  the  TEMTADS  measured  response  to  approximately  25 
milliseconds  versus  the  1.25  milliseconds  typically  measured  by  the  more  standard  EM61-MK2 
(EM61)  metal  detector. 

Data  collected  by  the  two  instruments  were  processed  using  the  UX- Analyze  add-on  to  Geosoft’s 
Oasis  montaj  processing  environment  to  determine  various  parameters  for  each  target.  The  primary 
parameters  of  concern  were  the  intrinsic  polarizability  (response  over  time)  of  each  axis  of  the 
source  object.  Because  the  polarizabilities  essentially  do  not  change  between  different  versions  of 
the  same  object,  the  polarizabilities  measured  for  an  unknown  source  object  at  a 
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field  site  can  be  compared  to  polarizabilities  measured  for  standard  objects  such  as  ordnance. 
UX- Analyze  was  also  used  to  determine  the  degree  of  match  between  field  data  points  and  site- 
specific  libraries  containing  known  polarizabilities  for  various  versions  of  the  ordnance  suspected 
to  be  present  at  each  site.  Following  library  matching,  each  target  was  classified  as  either  a  target 
of  interest  (TOI)  or  non-TOI  using  a  classification  scheme  developed  for  each  site.  A  ranked  dig 
list  was  submitted  to  ESTCP  for  each  site,  with  the  targets  most  likely  to  be  TOI  at  the  top  and  the 
targets  most  likely  to  be  considered  clutter  at  the  bottom. 

DEMONSTRATION  RESULTS 

The  standard  military  MR  work  conducted  (i.e.,  seeding,  dynamic  data  collection,  intrusive  work) 
on  the  projects  proceeded  without  issue.  There  was  little  of  note  with  these  activities  with  the 
exception  of  minor  changes  to  the  intrusive  procedures  as  the  projects  progressed,  to  streamline 
the  process  and  to  ensure  that  results  were  tracked  accurately.  The  projects  were  also  largely  very 
successful  with  regard  to  the  advanced  EMI  sensor  data  collection  and  classification  efforts.  Dig 
lists  successfully  identified  98%-100%  of  the  TOI  at  each  of  the  sites,  with  the  exceptions  of 
MMR,  where  the  TOI  detection  rate  was  as  low  as  83%  during  the  second  phase  of  intrusive 
investigation,  and  the  former  WMA,  where  it  was  concluded  that  all  detection  survey  targets  would 
need  to  be  excavated  to  remove  all  hazardous  material.  The  goal  of  the  project  at  MMR  was  more 
to  reduce  the  amount  of  large  TOI  at  the  site  than  to  identify  all  TOI  present,  so  the 
misclassification  of  some  TOI  was  not  a  significant  concern.  Classification  using  the  MetalMapper 
at  the  former  WMA  was  determined  to  be  ineffective,  primarily  due  to  the  excessive  geologic 
response  from  the  iron  rich  volcanic  environment.  Reductions  in  clutter  digs  ranged  from  about 
61%  at  Fort  Sill  to  about  91%  for  the  second  project  at  Camp  Beale,  with  the  exception  of  the 
former  WMA  MetalMapper  classification  study,  where  it  was  determined  that  no  reduction  was 
possible. 

IMPLEMENTATION  ISSUES 

The  single  largest  implementation  issue  on  any  of  the  projects  was  the  failure  of  one  of  the 
MetalMapper  transmitters  during  the  second  Camp  Beale  project.  Diagnosis  and  repair  of  this  issue 
required  about  130  hours  of  field  time  for  trouble-shooting  and  standby  time  at  the  beginning  of 
the  project.  A  transmitter  board  also  needed  to  be  replaced  during  the  Fort  Sill  project,  although 
diagnosis  and  repair  of  the  problem  did  not  take  nearly  as  long  as  the  additional  time  needed  at 
Camp  Beale.  Additionally,  the  serial-to-universal  serial  bus  (USB)  converter  that  was  originally 
shipped  with  the  MetalMapper  caused  repeated  software  crashes  during  both  the  first  Camp  Beale 
project  and  the  Fort  Sill  project.  Each  crash  necessitated  a  restart  of  the  collection  software  and  re¬ 
addition  of  the  points  to  be  collected,  which  severely  impacted  the  number  of  points  that  could  be 
collected  each  day.  Improved  versions  of  this  equipment  were  identified,  and  the  problem  has  been 
solved.  The  largest  implementation  issue  with  the  TEMTADS  was  a  re-shot  rate  nearing  25%.  The 
TEMTADS  survey  area  at  Camp  Beale  was  tree-covered  and  much  rockier  than  the  MetalMapper 
survey  area,  and  it  is  suspected  that  many  of  the  TEMTADS  targets  were  selected  based  on 
geophysical  noise  in  the  low-amplitude  EM61  data. 
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1.0  INTRODUCTION 


Currently,  up  to  90%  of  excavation  costs  on  most  unexploded  ordnance  (UXO)/munitions  and 
explosives  of  concern  (MEC)  projects  are  related  to  removing  scrap  metal  that  does  not  represent 
an  explosive  hazard.  Significant  cost  savings  could  be  achieved  through  the  use  of  geophysical 
discrimination  methods  that  could  reduce  the  number  of  excavations  required  to  remove  explosive 
hazards  from  sites.  Over  the  past  six  years,  the  Environmental  Security  Technology  Certification 
Program  (ESTCP)  has  been  conducting  discrimination  studies  at  active  and  Formerly  Used 
Defense  Sites  (FUDS)  across  the  country  to  demonstrate  the  effectiveness  of  advanced 
electromagnetic  induction  (EMI)  sensors  in  classifying  subsurface  metal  as  either  a  target  of 
interest  (TOI)  potentially  representing  MEC  that  should  be  intrusively  investigated,  or  as  clutter 
that  could  be  left  in  the  ground  without  further  investigation.  To  achieve  these  objectives, 
controlled  tests  were  conducted  at  the  Former  Camp  Beale,  CA;  Former  Pole  Mountain  Target  and 
Maneuver  Area  (PMTMA),  WY ;  Fort  Sill,  OK;  Massachusetts  Military  Reservation  (MMR),  MA; 
former  Camp  George  West,  CO;  and  former  Waikoloa  Maneuver  Area  (WMA),  HI,  as  part  of 
ESTCP  Munitions  Response  (MR)  projects  MR-201104  (Evaluation  and  Discrimination 
Technologies  and  Classification  Results)  and  ESTCP  MR-201157  (Demonstration  of 
MetalMapper  Static  Data  Acquisition  and  Data  Analysis). 

All  of  these  projects  involved:  the  seeding  of  project  sites  with  inert  ordnance  or  industry  standard 
objects  (ISOs)  intended  to  simulate  the  geophysical  characteristics  of  MEC,  the  use  or  collection 
of  EM61-MK2  (EM61)  detection  survey  data  to  identify  geophysical  anomalies,  the  collection  of 
cued  advanced  sensor  data  over  the  detection  survey  anomalies,  analysis  of  the  advanced  sensor 
data,  and  the  intrusive  investigation  of  the  anomalies.  Parsons  was  responsible  for  at  least  one  of 
these  activities  at  each  of  the  sites. 

1.1  BACKGROUND 

The  Fiscal  Year  2006  defense  appropriation  contained  funding  for  the  “Development  of  Advanced, 
Sophisticated  Discrimination  Technologies  for  UXO  Cleanup.”  ESTCP  responded  by  conducting 
a  UXO  discrimination  study  at  the  former  Camp  Sibert,  AL.  The  results  of  this  first  demonstration 
were  very  encouraging,  in  that  conditions  for  discrimination  were  favorable  at  this  site  and 
included  a  single  TOI  (4.2-inch  mortar)  and  benign  topography  and  geology.  All  of  the 
demonstrated  classification  approaches  correctly  identified  a  sizable  fraction  of  the  anomalies  as 
arising  from  nonhazardous  items  that  could  be  safely  left  in  the  ground.  Both  commercial  and 
advanced  sensors  produced  very  good  results.  ESTCP  organized  demonstrations  at  MR  sites  across 
the  country  between  2006  and  2015,  generally  with  new  variables  added  to  the  classification 
challenges  at  each  subsequent  site  (e.g.,  increased  target  density,  increased  response  from  local 
geology,  mixed  munition  sizes  ranging  from  small  to  very  large,  wooded  areas).  Additionally,  the 
subsequent  projects  included  the  use  of  smaller,  man-portable  EMI  sensors  such  as  the  Naval 
Research  Laboratory’s  Time-domain  Electromagnetic  Multi- sensor  Towed  Array  Detection 
System  (TEMTADS)  man-portable  2x2  cart  (TEMTADS),  Lawrence  Berkeley  National 
Laboratory’s  man-portable  Berkeley  UXO  discriminator,  and  Black  Tusk  Geophysics’  (BTG) 
man-portable  vector  (MPV)  machine.  All  of  the  EMI  sensors  tested  to  date  have  been  successful 
in  discriminating  between  TOI  and  clutter. 
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The  earlier  demonstration  projects — many  of  which  are  discussed  in  this  report — focused  on 
proving  that  the  technology  was  effective  by  comparing  theoretical  dig  lists  to  real-world  sources 
by  excavating  all  of  the  targets  at  a  given  site  and  comparing  the  known  source  results  to  the 
predicted  source  results.  More  recent  projects  have  focused  on  actually  leaving  metal  classified  as 
non-TOI  in  the  ground  following  the  completion  of  the  project.  The  second  ESTCP-funded  project 
at  the  former  Camp  Beale — also  discussed  in  this  report — and  a  non-ESTCP-related  removal 
action  performed  at  two  sites  at  the  former  Camp  Sibert  resulted  in  >7,000  dynamic  target  sources 
remaining  un-dug  at  both  sites,  with  regulatory  approval.  No  TOI  were  misclassified  at  either  site, 
and  before  the  addition  of  quality  assurance  verification  digs,  the  reduction  in  necessary  clutter 
digs  was  >90%  for  each. 

1.2  OBJECTIVES  OF  THE  DEMONSTRATIONS 

The  overall  objective  of  most  of  these  projects  was  to  demonstrate  the  ability  of  advanced  EMI 
sensors  and  associated  analysis  software  to  separate  subsurface  sources  likely  to  be  TOI,  from 
those  possibly  representing  MEC  from  sources  more  likely  to  be  metallic  clutter  that  did  not  pose 
an  explosive  hazard.  Other  activities  performed  during  these  projects,  such  as  the  seeding, 
detection  data  collection,  and  intrusive  investigation,  were  generally  performed  in  support  of  the 
overall  objective,  with  performance  objectives  for  those  portions  of  the  projects  governed  by 
Standard  Operating  Procedures  (SOPs)  previously  developed  on  numerous  Military  Munitions 
Response  Projects  (MMRPs).  The  one  exception  was  the  SOPs  for  the  intrusive  investigations, 
which  were  updated  through  the  course  of  the  demonstration  projects,  as  feedback  from  the 
intrusive  teams  regarding  the  process  flow  and  ESTCP  regarding  the  usability  of  the  results  were 
incorporated  into  the  procedures. 

In  addition  to  validating  the  performance  of  the  advanced  EMI  sensors,  the  second  Camp  Beale 
demonstration  project  (the  Camp  Beale  Pilot  Study)  was  intended  to  serve  as  a  vehicle  for  the  U.S. 
Army  Corps  of  Engineers  (USACE)  to  determine  the  most  effective  way  to  contract  a  project  that 
used  a  classification  approach  in  a  commercial  setting  as  opposed  to  the  demonstration  setting  used 
for  all  prior  ESTCP  projects.  The  project  also  allowed  USACE  to  coordinate  this  type  of  project 
with  a  local  regulator — in  this  case  the  California  Department  of  Toxic  Substances  Control 
(DTSC) — throughout  the  process.  Through  the  Technical  Project  Planning  process,  concurrence 
from  the  DTSC  was  obtained  on  all  decisions  made  for  each  phase  of  the  project. 
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2.0  TECHNOLOGY 


2.1  METALMAPPER 

The  MetalMapper  is  an  advanced  EMI  system  developed  by  Geometries,  Inc.,  with  support  from 
ESTCP.  The  MetalMapper  draws  elements  of  its  design  from  advanced  systems  currently  being 
developed  by  G&G  Sciences  Inc.  (supported  by  Naval  Sea  Systems  Command,  the  Strategic 
Environmental  Research  and  Development  Program  [SERDP],  and  ESTCP)  and  by  Lawrence 
Berkeley  National  Laboratory  (also  supported  by  SERDP  and  ESTCP).  It  has  three  mutually 
orthogonal  transmit  loops  in  the  Z,  Y,  and  X  directions  and  contains  seven  triaxial  receiver 
antennas  inside  the  Z  (bottom)  loop.  Typically,  the  transmit  loops  are  driven  with  a  classical 
bipolar  pulse-type  time  domain  electromagnetic  waveform  (i.e.,  alternating  pulse  polarity  with  a 
50%  duty-cycle).  Depending  on  the  survey  mode  (e.g.,  static/dynamic),  the  fundamental  frequency 
of  transmission  can  be  varied  over  the  range  l.ll<f<810  hertz  (Hz).  The  seven  receiver  antennas 
allow  21  independent  measurements  of  the  transient  secondary  magnetic  field. 

The  data  acquisition  computer  (DAQ)  is  built  around  a  commercially  available  product  from 
National  Instruments.  The  National  Instruments  DAQ  is  a  full-featured  personal  computer  running 
the  Windows  7  operating  system.  The  DAQ,  electromagnetic  transmitter,  and  batteries  for  the 
system  are  packaged  in  an  aluminum  case  that  can  be  mounted  on  a  pack  frame,  on  a  separate  cart 
such  as  a  hand  truck,  or  on  a  survey  vehicle  such  as  a  tractor.  The  instrumentation  package  also 
includes  two  external  modules  that  provide  real-time  kinematic  (RTK)  Global  Positioning  System 
(GPS)  location  and  platform  attitude  (i.e.,  magnetic  heading,  pitch,  and  roll)  data.  These  modules 
are  connected  to  the  DAQ  through  serial  RS232C  ports.  A  block  diagram  of  the  DAQ  system  is 
shown  in  Figure  2-1. 


A)  MM  DAQ  assembly  with  Panasonic  terminal. 


B)  Function  block  diagram  of  MM  system 


Figure  2-1.  DAQ  and  DAQ  Functional  Block  Diagram 

The  MetalMapper  has  two  modes  of  data  collection:  dynamic  and  static.  Data  collected  in  dynamic 
mode  results  in  data  files  containing  many  data  samples.  Generally  speaking,  dynamic  mode  data 
are  collected  while  the  antenna  platform  is  in  motion.  Static  mode  data  collection  is  employed  for 
cued  surveys.  As  its  name  implies,  the  antenna  platform  remains  static  or  motionless  during  the 
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period  of  data  acquisition.  Depending  on  the  acquisition  parameters  (e.g.,  sample  period  and 
stacking  parameter)  it  can  take  tens  of  seconds  to  complete  a  static  measurement.  The  results  of 
the  static  measurement  are  written  into  a  binary  data  file  containing  only  a  single  data  point 
representing  the  average  (stacked)  result,  usually  over  tens  or  even  hundreds  of  repetitions  of  the 
transmitter’s  base  frequency. 

Data  are  acquired  in  time  blocks  that  consist  of  a  fixed  number  of  transmitter  cycle  “repeats.”  Both 
the  period  and  the  repeat  factor  are  operator  selectable  and  are  varied  in  multiplicative  factors  of 
3.  The  MetalMapper  also  averages  an  operator- specified  number  of  acquisition  blocks  (NStack) 
before  the  acquired  data  are  saved  to  disk.  The  decay  transients  that  are  received  during  the  off- 
times  are  stacked  (averaged)  with  appropriate  sign  changes  for  positive  and  negative  half-cycles. 
The  decays  in  an  individual  acquisition  block  are  stacked,  and  the  decays  in  that  block  are  averaged 
with  other  acquisition  blocks  (assuming  the  operator  has  selected  NStack  >1).  The  resultant  data 
are  saved  as  a  data  point.  Figure  2-2  shows  the  typical  configuration  of  the  instrument  used  for 
collecting  cued  data. 


Figure  2-2.  Antenna  Array  and  Typical  Deployment  of  the  MetalMapper  System 

In  its  present  (third  generation)  form,  the  MetalMapper  has  been  demonstrated  and  scored  at 
numerous  live  site  demonstrations  carried  out  by  ESTCP.  The  performance  of  the  MetalMapper  at 
these  sites  is  documented  in  formal  reports  issued  by  the  various  contractors  working  on  those 
projects. 

2.2  TEMTADS 

The  EMI  sensor  used  in  the  TEMTADS  array  is  based  on  the  Navy-funded  Advanced  Ordnance 
Locator  (AOL)  developed  by  G&G  Sciences  Inc.  The  AOL  consists  of  three  transmit  coils 
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arranged  in  a  1 -meter  cube.  The  TEMTADS  has  adopted  the  transmit  (Tx)  and  receive  (Rx) 
subsystems  of  this  sensor  directly,  converted  them  to  35-centimeter  (cm)-square  sensors  that  can 
be  assembled  in  a  variety  of  array  configurations,  and  made  minor  modifications  to  the  control  and 
DAQ  to  make  it  compatible  with  the  deployment  scheme. 

A  photograph  of  a  standard  TEMTADS  sensor  element  (as  used  in  the  ESTCP  MR  project 
MR-200601  array)  under  construction  is  shown  in  the  left  panel  of  Figure  2-3.  The  transmit  coil  is 
wound  around  the  outer  portion  of  the  form  and  is  35  cm  on  a  side.  The  25-cm  receive  coil  is 
wound  around  the  inner  part  of  the  form,  which  is  reinserted  into  the  outer  portion.  An  assembled 
sensor  with  the  top  and  bottom  caps  used  to  locate  the  sensor  in  the  array  is  shown  in  the  right 
panel  of  Figure  2-3. 


Figure  2-3.  Construction  Details  of  an  Individual  TEMTADS  EMI  Sensor 

In  addition  to  the  TEMTADS  5x5  array  developed  under  ESTCP  MR-200601,  the  TEMTADS 
man-portable  (MP)  2x2  cart  system  was  designed  and  built  using  the  same  sensor  elements.  After 
demonstration  of  the  MP  system  at  the  Aberdeen  Test  Center  Standardized  UXO  Test  Site  in 
August  2010,  revision  of  the  sensor  technology  was  indicated  for  the  MP  system  to  collect 
sufficient  data  over  an  anomaly.  A  modified  version  of  the  sensor  element  was  designed  and  built, 
replacing  the  single,  vertical  receiver  loop  of  the  original  coil  with  a  three-axis  receiver  cube.  These 
receiver  cubes  are  identical  in  design  to  those  used  in  the  AOL2  and  Geometries  MetalMapper 
(ESTCP  MR-200603)  system  with  dimensions  of  8  cm  rather  than  10  cm.  The  BTG  MPV2  system 
(ESTCP  MR-201005)  uses  an  array  of  five  similar  receiver  cubes  and  a  circular  transmitter  coil. 
The  new  sensor  elements  are  designed  to  have  the  same  form  factor  as  the  originals,  aiding  in 
system  integration.  A  new  coil  under  construction  is  shown  in  Figure  2-4. 
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Figure  2-4.  Individual  Updated  TEMTADS  EMI  Sensor  with  Three-Axis  Receiver 

Decay  data  are  collected  with  a  500  kilohertz  (kHz)  sample  rate  until  25  milliseconds  after  turn¬ 
off  of  the  excitation  pulse.  This  results  in  a  raw  decay  of  12,500  points — too  many  to  be  used 
practically.  These  raw  decay  measurements  are  grouped  into  121  logarithmically  spaced  “gates” 
with  center  times  ranging  from  23  microseconds  to  24.35  milliseconds,  with  5%  widths,  and  are 
saved  to  disk. 

2.3  SENSOR  ARRAY 

The  TEMTADS  comprises  four  individual  EMI  sensors  with  three-axis  receivers,  arranged  in  a 
2x2  array  as  shown  in  Figure  2-5.  The  center-to-center  distance  is  40  cm,  yielding  an  80-cm  x  80- 
cm  array.  A  picture  of  the  array  mounted  on  the  TEMTADS  man-portable  2x2  cart  platform  is 
shown  in  Figure  2-6. 


EM  Sensor 


Figure  2-5.  Sketch  of  the  EMI  Sensor  Array  Showing  the  Position  of  the  Four  Sensors 
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Figure  2-6.  TEMTADS  Man-Portable  2x2  Cart  Sensor  Platform 
2.4  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

2.4.1  Advantages 

The  primary  advantage  of  the  MetalMapper,  TEMTADS,  MPV,  and  other  advanced  EMI  sensors 
tested  as  part  of  the  ESTCP  demonstration  program  (e.g.,  Berkeley  UXO  Discriminator, 
TEMTADS  5x5,  man-portable  Berkeley  UXO  Discriminator)  is  the  ability  of  these  sensors  to 
successfully  differentiate  subsurface  TOI  from  subsurface  clutter.  The  demonstration  program  has 
successful  proven  that  these  sensors  are  typically  capable  of  reducing  the  number  of  intrusive  digs 
required  on  MMRP  sites  significantly.  Reductions  of  >90%  have  been  demonstrated  on  sites 
containing  ordnance  ranging  from  37-millimeter  (mm)  projectiles  to  5-inch  rockets  and  nearly 
everything  in  between.  When  functioning  correctly,  the  instruments  can  collect  anywhere  from 
150  to  400  points  per  day  depending  on  deployment  options  (man  portable  versus  vehicle 
deployed).  The  main  advantage  of  the  MetalMapper  with  regard  to  all  other  advanced  EMI  sensors 
is  that  it  is  currently  commercially  available.  The  TEMTADS  is  also  generally  available  from  the 
Naval  Research  Laboratory  with  enough  advanced  planning,  while  the  other  sensors  mentioned 
are  generally  only  used  by  the  organizations  that  developed  them.  Geometries  plans  to  develop  a 
commercially  available  TEMTADS  sometime  in  2015. 

2.4.2  Limitations 
MetalMapper 

The  greatest  limitation  of  the  MetalMapper  is  its  size — both  of  the  sensor  itself  and  of  the 
accompanying  computer,  screen,  and  cables.  The  system  is  designed  primarily  for  use  in  relatively 
flat,  open  fields  and  cannot  currently  be  used  very  effectively  in  wooded  areas.  Given  the  large 
size,  the  MetalMapper  generally  needs  to  be  transported  via  a  vehicle  like  a  tractor,  skid  steer,  or 
extendable-reach  forklift.  The  mostly  wooden  sled  used  for  many  of  the  demonstration  projects 
was  designed  to  fit  on  the  rear  three-point  hitch  of  many  small-  to  medium- sized  tractors.  Other 
versions  of  this  sled  have  an  adapter  designed  to  attach  to  the  loader  on  the  front  of  tractors. 
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All  available  versions  of  this  sled  are  owned  by  various  USACE  offices,  and  it  is  generally  unknown 
which  version  of  the  adapter  will  be  available  for  a  given  project.  Parsons  planned  on  rear  mounting 
the  sled  used  for  another  classification  project  at  Camp  Sibert,  AL,  but  received  a  damaged  front- 
mounting  version  that  did  not  fit  on  the  loader  of  the  tractor  that  had  been  rented  for  the  project.  The 
use  of  a  tractor  rented  for  a  project  at  the  Hawthorne  Army  Depot  in  Nevada  proved  to  be  completely 
unfeasible  given  the  site  conditions  (beach  sand  with  dunes  that  the  tractor  could  not  navigate  safely), 
and  two  days  were  spent  locating  another  usable  transport  vehicle.  Another  mostly  fiberglass  sled 
has  been  designed,  but  that  one  will  only  fit  on  a  skid  steer  or  a  large  vehicle  capable  of  being 
mounted  with  a  skid  steer  adapter.  Available  sled  designs  and  transport  vehicles  need  to  be  identified 
well  before  the  field  team  is  deployed,  with  significant  thought  given  to  the  applicability  of  the 
vehicle  to  a  given  site.  Because  the  TEMTADS  is  man  portable,  there  are  far  fewer  transport  issues, 
although  the  operator-wom  backpack  is  not  light  and  consideration  needs  to  be  given  to  site 
conditions  with  regard  to  the  operators. 

In  addition  to  the  transport  concerns,  the  MetalMapper  requires  at  least  two  12- volt  batteries  to 
run,  and  three  if  the  transport  vehicle’s  battery  cannot  be  used  to  power  the  screen  and  the  inertial 
measurement  unit  (IMU).  There  were  no  power  outlets  available  for  overnight  changing  at  the 
former  Camp  Beale  site,  so  the  team  needed  two  extra  batteries  and  a  gas-powered  generator  that 
could  be  used  to  charge  the  second  set  of  batteries  during  the  day  while  the  other  set  was  in  use. 

TEMTADS 

The  largest  TEMTADS-specific  drawback  was  the  lack  of  a  real-time  method  for  positioning  the 
sensor  relative  to  the  source  in  the  field,  as  with  the  “dancing  arrows”  on  the  MetalMapper  display. 
The  operators  were  limited  to  placing  the  sensor  on  a  flag  located  by  other  means,  such  as  a 
reacquisition  team  with  an  EM61.  There  was  no  way  to  determine  whether  the  TEMTADS  point 
was  collected  over  the  source  without  a  full  inversion  in  UX- Analyze.  While  this  is  also  the  case 
with  data  collected  with  the  MetalMapper,  the  dancing  arrows  at  least  offer  an  indication  that  the 
sensor  is  in  the  correct  place  relative  to  the  source.  The  TEMTADS  has  been  upgraded  for  use 
with  an  in-field  quality  control  (QC)  program  that  performs  a  relatively  quick  inversion  after  a 
data  point  is  collected  after  Parsons’  work  at  Camp  Beale.  If  the  in-field  inversion  suggests  that 
the  original  point  was  not  collected  over  a  source,  the  program  directs  the  operator  to  the  location 
of  the  nearest  source,  and  a  second  point  can  be  collected  without  waiting  to  be  notified  by  the 
data  processor.  The  TEMTADS  can  also  now  be  integrated  with  a  GPS  or  robotic  total  station  for 
more  accurate  point  location  in  the  field. 

Finally,  all  of  these  sensors  can  be  extremely  difficult  to  repair  in  the  field.  As  an  example,  a 
transmitter  failed  on  different  MetalMapper  systems  during  both  the  Fort  Sill  demonstration  and 
the  Camp  Beale  Pilot  Study.  The  problem  was  diagnosed  in  the  field  in  both  cases,  but  only  the 
Fort  Sill  issue  could  be  fixed  in  the  field.  Luckily,  Camp  Beale  was  within  driving  distance  of  the 
Geometries  office  in  San  Jose,  and  the  time  spent  on  transport  and  repair  was  limited.  Various 
wrappings  of  the  triaxial  receivers  have  failed  on  other  projects  as  well.  To  date  on  Parsons’ 
projects,  all  failures  have  been  for  wrappings  on  outer  receivers,  and  the  data  was  not  significantly 
affected  as  long  as  the  data  from  the  malfunctioning  wrapping  was  not  used  in  inversion.  It  is 
assumed  this  would  have  been  a  much  larger  problem  had  the  issue  been  with  one  of  the  three 
middle  receivers.  It  seems  that  data  collection  could  be  put  on  hold  for  a  week  or  more  if  the  sensor 
needs  to  be  shipped  to  Geometries  from  a  more  remote  site. 
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3.0  PERFORMANCE  OBJECTIVES 


The  data  collection  and  analysis  objectives  for  the  various  demonstration  projects  are  summarized 
in  Table  3-1.  More  detail  regarding  the  performance  objective  tests  and  success  criteria  can  be 
found  in  the  Demonstration  Reports  for  each  site. 

Table  3-1.  Performance  Objectives  for  this  Demonstration  Projects 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

EM61  Detection  Data  Collection  Objectives 

Complete  coverage  of 
the  demonstration  site 

Along  line  point  spacing 
Survey  coverage 

Mapped  survey  data 

98%  of  data  points 
<25  cm  along  line 
>98%  coverage  at 

0.5-meter  line  spacing 

Repeatability  of 
instrument  verification 
strip  (TVS) 
measurements 

Amplitude  of 
electromagnetic  anomaly 
Measured  target  locations 

Twice-daily  IVS  survey 
data 

Amplitudes  +25% 

Location  ±25  cm 

Detection  of  all  TOI 

Percent  detected  of  seeded 
items 

Location  of  seeded  items 
Anomaly  list 

100%  of  seeded  items 
detected 

Reacquisition  Objectives  (TEMTADS  targets  only) 

GPS/Trimble  Robotic 
Total  Station  accuracy 

Difference  between 
measured  and  known 
monument  locations 

Known  location  of  QC 
point 

Measured  location  of  QC 
point 

100%  of  tests  within 

10  cm  of  expected 
location 

EM61  static  test 
repeatability 

Difference  between 
measured  Ch2  response 
and  standard  Ch2  response 

Standard  response  for  test 
object 

Measured  response  for  test 
object 

100%  of  tests  within 
+10%  of  standard 

Correctly  locate  flag 
locations 

Location  of  placed  flag 
relative  to  dynamic  EM61 
target 

Location  of  EM61  target 
GPS-located  position  of 
reacquired  location 

All  flagged  locations 
>30  cm  from  picked 
location  will  be  re¬ 
surveyed;  flagged 
locations  >73  cm  from 
picked  location  will  be 
flagged  at  both  picked  and 
reacquired  locations 

Cued  Data  Collection/Processing  Objectives 

Correctly  identify  seed 
items  in  IV S 

Percentage  of  IV S  items 
identified  correctly 

Twice  daily  IVS  survey 
data 

100%  of  IVS  items 
identified  correctly  with 
confidence  metric  of  >0.7 
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Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Correctly  position 
sensor  relative  to 

source 

Distance  between 
collection  location  and 
inverted  target  location 

GPS-located  collection 
location 

Modeled  target  location 

100%  of  inverted 
locations  within  40  cm  of 
collection  point  unless  re¬ 
shot  also  outside  radius 1 

Cued  Data  Collection/Processing  Objectives 

Correctly  position 
sensor  relative  to  EM61 
target 

(MetalMapper  only) 2 

Distance  between 
MetalMapper  collection 
location  and  EM61  target 
location 

GPS-located  collection 
location 

EM61  target  location 

100%  of  collection  points 
within  73  cm  of  EM61 
target  location2 

Cued  Data  Analysis  and  Classification  Objectives 

Maximize  correct 
classification  of  TOI 

Percentage  of  TOI  retained 

Prioritized  dig  lists 

Scoring  reports  from 
Institute  for  Defense 
Analyses 

All  TOI  classified 
correctly 

Maximize  correct 
classification  of  non- 
TOI 

Percentage  of  false  alarms 
eliminated 

Prioritized  dig  lists 

Scoring  reports  from 

Institute  for  Defense 
Analyses 

50%  of  non-TOI  left  in 
ground 

Correctly  identify  type 
of  TOI 3 

Percentage  of  TOI 
correctly  identified  by 
group 

Prioritized  dig  lists 

Scoring  reports  from 

Institute  for  Defense 
Analyses 

75%  of  TOI  identified 
correctly 

Correctly  identify  type 
of  non-TOI 3 

Percentage  of  non-TOI 
correctly  identified  by 
group 

Prioritized  dig  lists 

Scoring  reports  from 

Institute  for  Defense 
Analyses 

50%  of  non-TOI  ! 

identified  correctly 

Minimize  number  of 
“Can’t  Analyze” 
targets 

Percentage  of  targets  that 
cannot  be  classified  due  to 
questionable  data 

Demonstrator  target 
parameters 

Less  than  2%  of  points 
identified  as  “Can’t 
Analyze” 

Correct  estimation  of 
target  location 

Accuracy  of  estimated 
target  parameters  for  dig 
list  targets  marked  as  “dig” 

Demonstrator  target 
parameters 

Results  of  intrusive 
investigation 

MetalMapper 

X,  Y  <15  cm  (la) 

Z  <10  cm  (la) 

TEMTADS 

X,  Y  <60  cm  (la) 

Z  <15  cm  (la) 

ct  □  Standard  deviation 


1  In  addition  to  targets  with  both  initial  and  re-shot  inverted  locations  >40  cm  from  the  collection  point,  targets 
collected  specifically  to  be  within  73  cm  of  an  EM61  pick  location  did  not  negatively  affect  this  objective  if  inverted 
locations  were  >40  cm  from  the  collection  point. 

2  All  TEMTADS  points  were  collected  on  a  reacquired  EM61  point.  MetalMapper  points  with  no  identified  problems 
(e.g.,  inversion  offset,  noise)  and  no  other  targets  within  2  meters  of  the  target  in  question  were  not  re-collected. 

3  The  requirement  to  identify  types  was  only  applicable  for  the  Camp  Beale  Pilot  Study.  See  Demonstration  Report 
(Parsons,  2014)  for  further  information  regarding  groups. 
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4.0  SITE  DESCRIPTIONS 


Demonstration  project  locations  included  the  Former  Camp  Beale,  CA;  Former  PMTMA,  WY; 
Fort  Sill,  OK;  MMR,  MA;  former  Camp  George  West,  CO;  and  former  WMA,  HI. 

4.1  SITE  SELECTION 

Demonstration  sites  were  generally  selected  in  an  attempt  to  increase  the  complexity  of  the 
classification  process.  The  first  site  in  the  series,  former  Camp  Sibert  in  Alabama,  had  only  one 
TOI,  and  item  “size”  was  an  effective  discriminant.  A  hillside  range  at  the  former  Camp  San  Luis 
Obispo  in  California  was  selected  for  the  second  of  these  demonstrations  because  of  the  wider  mix 
of  munitions,  including  60-mm,  81-mm,  4.2-inch  mortars,  and  2.36-inch  rockets.  The  third  site 
chosen  was  the  former  Camp  Butner  in  North  Carolina,  which  was  known  to  be  contaminated  with 
items  as  small  as  37-mm  projectiles,  adding  yet  another  layer  of  complexity  into  the  process.  The 
sites  completed  under  Parsons’  demonstration  projects  were  selected  as  demonstration  sites  for  the 
following  reasons: 

•  Camp  Beale,  CA:  varying  geology  across  site,  only  wooded  areas  that  could  only  be 
surveyed  using  man-portable  sensors,  suspected  munitions  ranging  from  37  mm  to  105  mm 

•  PMTMA,  WY :  wide  range  of  munitions,  ongoing  project  with  available  data  set  collected 
by  another  contractor,  not  a  significant  upgrade  in  difficulty  from  Camp  Beale 

•  Fort  Sill,  OK:  active  base,  large  list  of  possible  munitions  ranging  in  size  from  very  small 
(fragmentation  balls  from  40-mm  grenades)  to  fairly  large  (3.5-inch  rockets),  areas  of 
extremely  high  anomaly  density 

•  MMR,  MA:  anomaly  density  as  high  or  higher  than  Fort  Sill,  difficult  terrain  for  data 
collection  (stumps,  vegetation,  impact  craters) 

•  Camp  George  West,  CO:  very  steep  hillside  that  could  only  be  reached  with  man-portable 
sensors  (intrusive  investigation  only) 

•  WMA,  HI:  extremely  responsive  geology  with  background  values  varying  significantly 
across  small  areas,  extremely  rocky  terrain  for  MetalMapper  project;  intrusive 
investigation  for  MPV  surveys  performed  in  area  with  varying  amounts  of  cultural  features 

4.2  GEOLOGY 

Detailed  descriptions  of  site  geology  are  contained  in  the  Demonstration  Reports  for  each  site.  In 
general,  the  only  sites  where  geology  had  a  significant  effect  on  the  advanced  sensor  data  were  the 
former  Camp  Beale  and  the  former  WMA.  Geologic  effects  were  noticeable  at  the  former  Camp 
Beale,  but  they  could  mostly  be  rendered  negligible  with  the  collection  and  application  of  the 
appropriate  background  data.  A  detailed  discussion  of  the  background  issues  noted  at  the  former 
Camp  Beale  is  contained  in  the  Pilot  Study  Report  (Parsons,  2012).  Geologic  effects  were  much 
more  apparent  at  the  former  WMA  given  the  site’s  location  on  an  iron-rich  volcanic  lava  with  very 
little  soil  cover.  Significant  discussion  of  the  site  geology  and  its  effects  on  the  classification  effort 
are  contained  in  the  MetalMapper  and  MPV  Demonstration  Reports  for  the  former  WMA 
(Parsons,  2015  and  2016). 
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4.3  MUNITIONS  CONTAMINATION 

The  following  munitions  were  considered  TOI  at  one  or  more  of  the  demonstration  sites: 

•  20-mm  cartridges  (Fort  Sill) 

•  37-mm  projectiles  (Camp  Beale,  PMTMA,  Fort  Sill,  MMR,  WMA) 

•  40-mm  grenade  fragmentation  balls  with  high  explosive  (HE)  filler  (Fort  Sill) 

•  40-mm  grenades  (Fort  Sill) 

•  75-mm  projectiles  (Camp  Beale,  PMTMA,  Fort  Sill,  Camp  George  West,  MMR,  WMA) 

•  3-inch  projectiles  (PMTMA) 

•  105-mm  projectiles  (Camp  Beale,  PMTMA,  MMR,  WMA) 

•  155-mm  projectiles  (PMTMA,  MMR,  WMA) 

•  60-mm  mortars  (Camp  Beale,  Fort  Sill,  MMR,  WMA) 

•  81 -mm  mortars  (Camp  Beale,  MMR,  WMA) 

•  4.2-inch  mortars  (MMR) 

•  66-mm  light  antitank  rockets  (Fort  Sill) 

•  2.36-inch  rockets  (Fort  Sill,  MMR,  WMA 

•  2.75-inch  rockets  (MMR) 

•  3-inch  Stokes  mortars  (PMTMA) 

•  3.5-inch  rockets  (Fort  Sill,  MMR) 

•  Antitank  land  mines  (Fort  Sill,  WMA) 

•  Hand  grenades  (Fort  Sill,  WMA) 

4.4  SITE  CONFIGURATION 

Detailed  descriptions  of  and  maps  showing  the  areas  investigated  at  the  demonstration  sites  are 
contained  in  the  Demonstration  Reports  for  each  site.  The  following  is  a  brief  summary  of  the 
acreages  and  targets  surveyed  at  each  site,  where  applicable: 

•  Former  Camp  Beale  Demonstration:  9  acres  of  50-acre  site;  split  into  MetalMapper 
(6  acres)  and  man-portable  sensor  (3  acres)  areas;  1,490  MetalMapper  targets 

•  Former  Camp  Beale  Pilot  Study:  27  acres  of  50-acre  site;  split  into  MetalMapper 
(17.6  acres)  and  TEMTADS  (9.4  acres)  areas;  6,363  MetalMapper  targets  and  2,806 
TEMTADS  targets 

•  Former  PMTMA:  Analysis  of  2,370  MetalMapper  targets  collected  over  a  10-acre  area  by 
Sky  Research,  Inc.;  no  data  collection 

•  Fort  Sill:  6  acres;  1,988  MetalMapper  targets 

•  MMR:  6  acres;  split  into  MetalMapper  and  TEMTADS  areas  (3  acres  each);  2,287 
MetalMapper  targets 

•  Former  Camp  George  West:  2  acres;  intrusive  investigation  of  466  targets  collected  and 
classified  by  Sky  Research 

•  Former  WMA:  5.3  acres;  1,031  MetalMapper  targets. 

•  Former  WMA:  477  intrusive  investigation  of  477  MPV  targets  identified  by  BTG 
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5.0  TEST  DESIGN 


5.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

To  varying  degrees,  these  projects  were  conducted  with  two  primary  objectives: 

•  Test  and  validate  detection  and  discrimination  capabilities  of  currently  available  and 
emerging  advanced  EMI  sensors  developed  specifically  for  discrimination  on  real  sites 
under  operational  conditions. 

•  Investigate  in  cooperation  with  regulators  and  program  managers  how  classification 
technologies  can  be  implemented  in  MEC  cleanup  operations. 

The  key  components  of  this  method  were: 

•  collecting  EM61  data  across  each  site  and  selecting  anomalous  areas  in  that  data; 

•  seeding  the  sites  to  determine  the  effectiveness  of  the  advanced  sensors; 

•  collecting  advanced  sensor  data; 

•  analyzing  the  advanced  sensor  data  using  physics-based  models  to  extract  target  parameters 
such  as  size,  shape,  and  materials  properties; 

•  using  those  parameters  to  construct  a  ranked  anomaly  list; 

•  selecting  verification  digs  from  the  subset  of  targets  classified  as  non-digs  on  the  ranked 
dig  list;  and 

•  intrusively  investigating  the  dig  and  verification  targets. 

Whenever  possible,  blind  seeding  for  both  EM61  and  advanced  sensor  QC  was  performed  before 
EM61  data  collection  so  targets  for  the  seeds  would  be  present  in  that  data.  Advanced  sensor  cued 
data  were  collected  over  targets  selected  from  the  EM61  data,  and  the  resulting  data  were  analyzed 
by  one  or  more  demonstrators.  Demonstrators  processed  the  individual  cued  data  sets  to  extract 
various  parameters  for  each  of  the  targets.  Classifiers  were  developed  based  on  the  resulting 
parameters  and  a  very  limited  set  of  site-specific  ground  truth,  and  each  demonstrator  submitted 
one  or  more  ranked  dig  lists  for  each  site. 

The  ranked  dig  lists  were  submitted  with  all  targets  used  as  ground  truth  at  the  top  (“Must  Dig”) 
followed  immediately  by  targets  for  which  reliable  parameters  could  not  be  extracted  (“Can’t 
Analyze”).  The  remaining  targets  were  ranked  from  most  likely  to  be  TOI  to  most  likely  to  be 
clutter,  with  a  stop-dig  point  identified  at  a  specific  point  in  the  dig  list  (all  targets  above  this  point 
would  be  dug,  all  below  would  be  left  in  the  ground).  In  later  demonstrations,  it  was  also  required 
that  each  point  be  identified  with  an  approximate  size  (TOI)  and  a  size  and  shape  (non-TOI).  These 
inputs  were  scored  by  the  Institute  for  Defense  Analysis  with  emphasis  on  the  number  of  items 
correctly  labeled  nonhazardous  while  correctly  labeling  all  TOI. 

Verification  digging  typically  included  the  intrusive  investigation  of  all  anomalies  on  the  master 
dig  list,  again  with  the  exception  of  the  Camp  Beale  Pilot  Study.  The  Pilot  Study  verification 
digging  consisted  of  only  a  relatively  small  portion  of  the  targets  classified  as  non-TOI  on  the 
single  dig  list  submitted  for  the  project.  The  verification  digs  for  that  project  were  selected  from 
the  non-TOI  list  by  USACE  personnel.  For  all  intrusive  investigations,  each  target  was  uncovered, 
photographed,  located  with  a  cm-level  GPS,  and  removed. 
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5.2  SITE  PREPARATION 


5.2.1  Brush  Cutting/Surface  Clearance 

Brush  cutting  and  surface  sweeps  were  performed  as  necessary  before  collection  of  the  EM61  data 
used  to  select  targets  for  the  advanced  EMI  surveys.  The  main  objective  of  the  surface  clearances 
was  to  ensure  that  no  hazardous  items  would  be  encountered  prior  to  the  nonintrusive  phases  in 
the  demonstration  area  and  to  remove  metallic  surface  debris  from  the  grids. 

5.2.2  Initial  EM61-MK2  100%  Coverage  Survey 

EM61  surveys  were  performed,  as  necessary,  before  the  collection  of  advanced  EMI  data  and  were 
used  to  identify  targets  for  the  advanced  EMI  surveys.  In  many  cases,  preexisting  EM61  data 
collected  as  part  of  another  ongoing  project  were  leveraged  for  use  on  the  demonstration  projects. 
For  those  sites  where  EM61  data  were  collected  specifically  for  the  demonstration  project,  data 
were  collected  along  parallel  lines  spaced  50  cm  apart,  with  location  typically  determined  using 
RTK  GPS  or  robotic  total  station  units.  Fiducials  were  used  in  limited  cases  in  thickly  wooded 
areas.  The  EM61  was  set  up  in  the  four-channel  mode. 

5.2.3  Seeding  Operations 

At  most  live  sites,  the  ratio  of  clutter  to  TOI  is  such  that  only  a  small  number  of  TOI  may  be  found 
in  the  investigation  area — far  from  enough  to  determine  classification  performance  with  acceptable 
confidence  bounds.  To  avoid  this  problem,  sites  were  seeded  with  enough  TOI  to  ensure 
reasonable  statistics. 

Proposed  seed  locations  were  flagged  using  either  GPS  or  robotic  total  station  units,  and  anomaly 
avoidance  was  practiced  at  each  location  to  ensure  a  clean  area  for  emplacement.  Seed  locations  were 
dug  to  the  size  and  depth  specified  in  the  seeding  plan  generated  for  each  site.  Intrusive  operations 
involved  both  mechanical  and  manual  procedures  to  meet  exact  specifications  and  to  minimize  burial 
evidence.  Prior  to  emplacement,  magnetic  north  was  determined.  Once  magnetic  north  was 
established,  the  seed  item  was  positioned  with  the  nose  pointing  to  the  exact  azimuth  and  dip  angle 
specified  by  Parsons.  The  dip  angle  specifications  were  set  to  a  45-degree  tolerance,  with  the  exact 
angles  measured  with  a  level.  Exact  angles  above  horizontal  and  below  horizontal  were  recorded. 
After  all  the  emplacement  requirements  of  depth,  inclination,  dip  angle,  length,  and  location  were 
completed,  a  photo  was  taken  of  the  seed  item  in  the  burial  location.  All  the  emplacement  information 
along  with  the  seed  item  and  north  direction  is  visible  in  the  photos. 

Seed  location  holes  were  not  backfilled  until  the  final  QC  check  was  completed.  This  consisted  of 
comparing  the  location  with  the  original  designated  location,  capturing  the  center  location  of  the 
emplaced  seed  item  with  GPS,  and  checking  the  depth,  inclination,  and  dip  angle  of  each  seed 
item.  Once  these  checks  were  accomplished,  the  backfilling  was  started  with  a  shovel  to  prevent 
any  excess  movement  of  the  seed  items. 

5.2.4  Instrument  Verification  Strips  (IVSs) 

One  or  more  instrument  verification  strips  (IVSs)  were  constructed  at  each  of  the  project  sites 
where  advanced  sensor  data  were  collected.  IVSs  generally  contained  four  or  five  ISOs  or  inert 
munitions  of  the  type  suspected  at  each  respective  site  and  a  cleared  location  intended  to  be  used 
as  a  background  point  to  correct  the  IVS  data. 
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5.3  SYSTEM  SPECIFICATION 


5.3.1  MetalMapper 

The  MetalMapper  sensor  and  DAQ  are  described  in  detail  in  Section  2.1.  During  the  projects,  the 
antenna  array  was  placed  in  a  sled  attached  to  either  the  rear  three-point  hitch  of  a  tractor  or  to  the 
front  of  an  extendable  reach  forklift.  A  Trimble  R8  GPS  was  mounted  directly  above  the  sensor 
array  using  a  wooden  tripod,  and  an  IMU  was  attached  to  the  wooden  support  used  to  stabilize  the 
X-  and  Y-direction  transmitters,  also  directly  above  the  center  of  the  array.  These  instruments 
streamed  positional  data  constantly,  at  a  rate  of  approximately  10  Hz.  The  two  instruments  were 
connected  to  the  DAQ  via  universal  serial  bus  (USB)  ports,  and  the  incoming  data  were  used  both 
to  navigate  from  point  to  point  and  to  locate  the  collected  data. 

5.3.2  TEMTADS 

The  TEMTADS  sensor  and  data  acquisition  system  are  described  in  detail  in  Section  2.2.  During 
the  Camp  Beale  Pilot  Study,  the  antenna  array  was  placed  on  a  cart  that  could  be  pushed  by  one 
of  the  two  data  collection  team  members.  The  other  team  member  was  responsible  for  naming  data 
files  and  starting  data  collection  via  a  tablet  computer  wirelessly  connected  to  the  data  collection 
computer  in  the  backpack.  Points  were  collected  directly  on  top  of  flags  placed  by  a  reacquisition 
team  that  used  an  EM61  to  relocate  the  targets  identified  in  the  original  EM61  data  set.  All  targets 
were  located  according  to  a  local  X,  Y  coordinate  system  with  the  center  of  the  sensor  on  the  0,  0 
point.  Inverted  source  locations  were  translated  to  the  project  coordinate  system  during  post¬ 
processing. 

5.4  CALIBRATION  ACTIVITIES 

5.4.1  Test  Pit  and  Instrument  Verification  Strip  Data  Collection 

A  test  pit  or  test  stand  was  constructed  at  each  site  before  starting  data  collection.  The  pits  were 
deep  enough  to  fit  the  largest  test  item  both  horizontally  and  vertically.  The  items  placed  in  the 
test  pits  consisted  of  available  objects  that  were  either  suspected  to  be  present  onsite  or  were  known 
to  have  been  used  as  seed  items.  Test  pit  items  were  oriented  in  either  three  directions  (ISOs;  no 
difference  between  nose  up  and  nose  down,  oriented  horizontally,  vertically,  and  45  degrees)  or 
five  directions  (inert  ordnance;  oriented  horizontally,  vertically  nose  up  and  down,  45  degrees  nose 
up  and  down)  relative  to  the  sensor  being  tested  and  were  placed  at  depths  expected  to  produce  a 
strong  signal-to-noise  ratio  for  each  item. 

In  addition  to  the  test  pit  data,  data  were  collected  over  an  IVS  twice  daily,  before  and  after  daily 
data  collection.  All  data  collected  over  the  IVSs  were  inverted  in  the  field  as  described  in 
Section  6.2  and  compared  to  their  respective  target  libraries  as  described  in  Section  6.4.  Two  tests 
were  performed  using  the  IVS  data: 

1.  Inverted  locations  were  compared  to  the  known  locations  for  the  IVS  seed  items,  with  the 
differences  between  the  modeled  and  known  locations  expected  to  be  <15  cm  X,Y  and 
10  cm  depth. 

2.  The  item  identified  by  the  target  library  comparison  was  compared  to  the  actual  buried 
item,  and  it  was  expected  that  the  identified  item  matched  TOI  with  a  confidence  high 
enough  that  it  would  be  marked  as  a  dig  (0.7  confidence  expected  in  the  field). 
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5.4.2  Background  Data 

Background  data  were  generally  collected  at  least  every  two  hours.  MetalMapper  background 
collection  points  were  determined  by  the  operator  who  searched  for  a  clear  location  using  the 
dancing  arrows  display  on  the  computer  screen.  TEMTADS  background  collection  points  were 
identified  by  the  reacquisition  team  using  an  EM61  and  a  Schonstedt  magnetometer.  The 
reacquisition  team  flagged  clear  locations  for  later  collection  by  the  TEMTADS  team.  Background 
data  were  collected  more  than  once  every  two  hours  if  the  operator  felt  another  point  was  necessary 
for  any  reason  (e.g.,  moving  from  one  location  onsite  to  another,  changes  to  the  configuration  of 
the  instrument,  changing  field  conditions  such  as  rain). 

5.5  DATA  COLLECTION  PROCEDURES 

5.5.1  MetalMapper  Data  Collection 

The  operator  moved  the  array  by  lifting  the  sled,  navigating  to  the  vicinity  of  each  selected  point 
using  the  graphic  display  on  the  computer  monitor,  and  setting  the  MetalMapper  down  on  the 
point.  Reacquisition  of  the  EM61  targets  selected  for  cued  data  collection  was  accomplished  using 
dancing  arrows  displayed  on  the  monitor.  The  dancing  arrows  display  shows  the  seven  receivers 
in  the  array,  arranged  as  they  are  in  the  Z-coil,  typically  with  a  blue  arrow  pointing  out  of  each. 
The  arrows  point  toward  the  metallic  source  nearest  each  of  the  receivers.  Under  ideal  conditions, 
one  source  is  in  the  vicinity  of  the  selected  point,  and  all  of  the  arrows  point  inward  toward  the 
center  of  the  array.  In  the  case  of  multiple  sources,  one  or  more  of  the  outer  arrows  may  point 
outward  from  the  array  toward  another  piece  of  metal.  Generally,  the  operator  attempted  to  position 
the  array  such  that  the  arrows  in  the  three  receivers  closest  to  the  middle  of  the  coil  were  pointing 
at  each  other. 

The  MetalMapper  single-point  or  cued-collection  mode  was  used  for  all  data  collection.  Once  the 
MetalMapper  was  positioned  correctly  above  the  target,  the  operator  collected  a  data  point  using 
the  settings  indicated  in  Table  5-1. 

Table  5-1.  MetalMapper  Acquisition  Parameters 


Mode 

Tx 

Mode 

Hold-Off 
Time  (ps) 

Block 

Period(s) 

Rep 

Fctr 

Dec 

Fctr 

(%) 

Stk 

Const 

Base 

Freq 

(Hz) 

Decay 
Time  (ps) 

No. 

Gates 

Sample 
Period  (s) 

Sample 

Rate 

(S/s) 

Static 

ZYX 

50 

0.9 

27 

10 

10 

30 

8333 

50 

9 

N/A 

Static  targets  were  identified  according  to  the  identification  (ID)  determined  for  each  target  picked 
in  the  dynamic  EM61  survey.  For  repeated  measurements  associated  with  a  single  target  point, 
10,000  was  generally  added  to  the  original  ID.  The  standard  addition  for  repeats  during  the  Camp 
Beale  Pilot  Study  was  50,000  because  it  was  possible  that  more  than  10,000  targets  would  be 
collected  (For  example,  the  re-shot  for  0001  was  50001. 
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5.5.2  TEMTADS  Data  Collection 


Data  collection  was  performed  by  two  field  teams:  a  reacquisition  team  and  a  TEMTADS  team. 
The  reacquisition  team  consisted  of  a  geophysicist  and  a  UXO  Technician  II;  the  TEMTADS  team 
consisted  of  two  geophysicists.  The  UXO  technician  on  the  reacquisition  team  removed  any  metal 
identified  during  flag  placement  and  made  a  note  of  anything  removed  at  each  point  so  that 
removed  items  could  later  be  compared  to  TEMTADS  results.  The  two  TEMTADS  operators 
switched  between  pushing  the  cart  and  operating  the  tablet  computer  used  to  control  data  collection 
so  one  person  was  not  carrying  the  backpack  for  the  entire  day. 

The  TEMTADS  single-point  or  cued-collection  mode  was  used  for  all  data  collection  at  Camp 
Beale.  Once  the  TEMTADS  was  positioned  correctly  above  the  flag  placed  by  the  reacquisition 
team,  the  operator  collected  a  data  point  using  the  settings  indicated  in  Table  5-2. 

Table  5-2.  TEMTADS  Acquisition  Parameters  Used  at  Camp  Beale 


Mode 

Tx 

Mode 

Hold-Off 
Time  (ps) 

Block 

Period(s) 

Rep 

Fctr 

Dec  Fctr 

(%) 

Stk 

Const 

Base  Freq 
(kHz) 

Decay 
Time  (ms) 

No. 

Gates 

Sample 
Rate  (S/s) 

Static 

xxxx 

25 

0.9 

9 

5 

18 

500 

24.35 

122 

N/A 

5.5.3  Scale  of  Demonstration 

The  areas  covered  and  number  of  targets  surveyed  at  each  site  are  listed  in  Section  4.4. 

5.5.4  Sample  Density 

One  data  point  was  collected  per  target,  as  described  in  Sections  5.5.1  and  5.5.2;  re-shots  were 
collected  for  targets  with  modeled  locations  >40  cm  from  the  collection  location  for  either  sensor. 
Re-shots  were  also  collected  for  MetalMapper  collection  locations  farther  than  an  expected 
distance  from  the  EM61  pick,  as  determined  using  the  standard  EM61  to  seed  item  data  quality 
objective  (DQO)  of  one-half-line  spacing  plus  35  cm. 

5.5.5  Data  Quality  Checks 

An  instrument  calibration  check  was  conducted  at  the  IVS  a  minimum  of  twice  a  day  (at  the 
beginning  and  the  end  of  the  field  day).  The  data  were  inverted  and  checked  against  the  items  in 
the  IVS  for  type  and  location.  These  checks  ensured  that  the  instrumentation  was  functional, 
properly  calibrated,  and  stable. 

Quality  checks  of  all  static  data  points  were  performed  after  initial  inversion  was  completed  using 
the  UX- Analyze  module  in  Oasis  montaj.  Inverted  target  locations  were  compared  to  data 
collection  locations  to  determine  if  offsets  between  the  two  were  >40  cm.  The  MetalMapper 
collection  locations  were  also  compared  to  the  location  of  the  EM61  targets.  Re-shots  were 
collected  for  targets  with  collection  to  modeled  locations  offsets  >40  cm  for  either  sensor,  or  EM61 
pick  to  collection  locations  distances  greater  than  a  maximum  expected  offset  based  on  the  EM61 
target  to  seed  item  DQO  for  the  dynamic  survey. 
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5.5.6  Data  Handling 

Collected  data  were  recorded  in  binary  format  as  files  on  the  hard  disk  of  the  DAQs.  These  data 
were  offloaded  to  other  media  at  least  once,  and  sometimes  more  frequently,  per  day.  The  computer 
hard  disks  had  enough  capacity  to  store  all  the  data  from  the  entire  site,  so  these  data  were  not 
erased  until  they  had  been  thoroughly  reviewed  and  archived.  The  data  file  names  acquired  each 
day  were  cataloged  (usually  on  a  spreadsheet)  and  integrated  with  any  notes  or  comments  in  the 
operator’s  field  book.  All  data  were  stored  on  the  hard  drive(s)  of  one  or  more  laptop  computers 
used  to  post-process  data.  Data  were  also  archived  to  a  data  server  in  the  Parsons  office. 

Raw,  binary  MetalMapper  files  were  preprocessed  using  the  TEM2CSV  software  package,  which 
outputs  “preprocessed  and  located”  data  files  in  a  text  readable  format  (.csv).  Preprocessing 
included  the  location  of  the  point  in  Universal  Transverse  Mercator  (UTM)  meters  and  subtraction 
of  background.  Located  and  background  corrected  .csv  files  were  imported  into  Oasis  montaj  for 
further  processing  and  analysis.  Raw  TEMTADS  files  were  converted  to  .csv  files  using  the  TEM 
Data  Logger  program  used  to  collect  the  data.  Unlike  the  MetalMapper  pre-processing, 
TEMTADS  .csv  files  were  not  translated  to  UTM  coordinates  and  were  not  background  corrected. 
Background  correction  was  accomplished  using  the  Level  Advanced  Sensor  Data  tool  in  UX- 
Analyze,  and  translation  was  performed  in  the  project’s  Microsoft  Access  database  after  most 
classification  had  already  been  performed. 

5.6  INTRUSIVE  PROCEDURES 

Reacquisition  of  all  targets  was  conducted  using  a  Trimble®  R8  GPS  system.  Parsons  flagged  all 
target  locations  with  a  plastic  pin  flag  that  was  marked  with  the  target  ID  and  EM61  pre-value. 
Location  data  captured  by  GPS  was  used  to  document  the  center  mass  of  each  item;  tape  measures 
were  used  to  measure  the  depth  to  the  center  of  mass  of  each  item  from  the  ground  surface.  A 
photograph  was  collected  of  the  item  with  written  dig  result  data  shown  on  a  whiteboard.  Lastly, 
an  EM61  was  used  to  scan  the  location  to  confirm  the  absence  of  all  metallic  items  from  that  target 
location. 

The  Parsons  team  leader  orchestrated  the  movements  of  the  different  tasks  associated  with  the 
information-gathering  process.  Llags  were  initially  placed  in  the  proper  location  as  indicated  by  a 
GPS.  An  operator  with  an  EM61  would  then  find  the  EM61  peak  nearest  the  flag,  move  the  flag 
to  that  location,  and  write  the  initial  EM61  response  on  the  flag.  Subsequent  intrusive  operations 
consisted  of  two  teams,  with  each  responsible  for  the  intrusive  work  and  white  board 
documentation  of  their  respective  holes.  A  single  data  collection  specialist  then  transcribed  all 
white  board  and  pre-  and  post-excavation  EM61  values  to  a  paper  dig  sheet. 

Munitions  debris  and  cultural  debris  scrap  collected  from  target  locations  were  stored  in  plastic 
buckets  in  the  field  and  transferred  to  the  storage  conex  at  the  end  of  each  day.  Parsons’  senior 
UXO  and  site  safety  officer  certified  all  munitions  debris  scrap  by  examining  each  piece 
individually  before  final  disposition  in  the  sealed  bins.  All  seed  items  recovered  from  intrusive 
operations  were  stored  in  a  secure  area,  prepared  for  final  shipment,  and  returned  to  the  entity  that 
supplied  them  at  the  completion  of  the  project.  Shipments  of  inert  ordnance  included  certified  and 
signed  U.S.  Department  of  Defense  1348  forms. 
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All  target  locations  were  backfilled  after  completion  of  the  excavation.  Once  the  final  targets  were 
excavated  and  backfilled,  Parsons  conducted  a  walkthrough  and  confirmed  that  all  holes  were 
filled  and  no  trash  was  left  behind. 

Excavation  data  collected  by  the  intrusive  team  was  entered  into  the  project  Microsoft  Access 
database  and  reviewed  daily.  The  daily  information  required  the  target  ID  to  be  connected  with 
intrusive  documentation,  photo,  and  GPS  coordinates.  Assessment  of  each  target  item  required  the 
coordinates  to  match  the  original  location  and  the  picture  to  match  the  documented  findings. 
Intrusive  results  were  also  compared  with  predicted  classification  results.  Intrusive  results  that 
disagreed  significantly  with  predicted  results  (e.g.,  advanced  sensor  data  that  suggested  the 
presence  of  TOI  with  high  confidence  versus  nothing  found)  were  flagged  for  recheck. 
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6.0  DATA  ANALYSIS  AND  PRODUCTS 


The  processing  and  analysis  steps  that  were  used  to  generate  a  dig/no  dig  decision  for  each  target 
are  described  below. 

6.1  PREPROCESSING 

6.1.1  MetalMapper  Preprocessing 

The  MetalMapper  acquisition  software  uses  a  convention  for  assigning  a  unique  name  to  each  data 
file  without  the  need  to  manually  enter  the  name.  The  operator  supplies  a  prefix  for  the  root  name 
of  the  file  (e.g.,  “Static”).  The  acquisition  software  then  automatically  appends  a  five-character 
numerical  index  to  the  filename  prefix  to  form  a  unique  root  name  for  the  data  file  (e.g., 
StaticOOOOl).  The  index  is  automatically  incremented  after  the  file  has  been  successfully  written. 
Although  the  target  ID  is  not  used  as  the  file  name  in  the  .tern  file,  the  target  ID  is  stored  in  the  file 
according  to  the  name  of  the  target  highlighted  on  the  MetalMapper  screen  during  collection. 

Preprocessing  of  the  MetalMapper  .tern  files  was  accomplished  using  TEM2CSV,  a  program 
specifically  developed  for  this  purpose.  TEM2CSV  subtracted  the  site  background  from  the  data 
point  using  a  background  file  specified  by  the  user,  converted  the  points  from  the  geographic 
coordinate  system  used  for  collection  to  the  UTM  Zone  coordinate  system  used  for  processing, 
and  exported  the  resulting  data  to  a  .csv  file  that  could  be  imported  into  the  UX- Analyze  package 
in  Geosoft’s  Oasis  montaj  software.  The  exported  .csv  file  name  contained  both  the  collection  ID 
and  the  target  ID  (e.g.,  B2_00001_CBStatic000012_00001  for  target  1).  Background  files  were 
usually  collected  every  two  hours  in  a  geophysical  quiet  location.  Unless  there  appeared  to  be  a 
problem  with  a  background  file,  the  file  was  used  to  correct  the  data  collected  from  approximately 
one  hour  before  the  background  file  was  collected  to  one  hour  after  it  was  collected. 

6.1.2  TEMTADS  Preprocessing 

The  TEMTADS  acquisition  software  allows  naming  files  at  the  time  of  collection,  so  all 
TEMTADS  points  were  identified  using  the  EM61  target  ID  on  the  target  list.  Conversion  of  the 
raw  data  files  to  text  readable  .csv  files  was  accomplished  using  the  TEM  Datalogger  program 
used  to  collect  the  data  and  then  imported  into  Oasis  montaj  for  all  further  processing.  Background 
correction  for  each  point  was  accomplished  using  the  Level  Advanced  Sensor  Data  tool  in  UX- 
Analyze.  As  with  the  MetalMapper  data,  background  points  were  typically  used  to  correct  data 
points  collected  one  hour  before  to  one  hour  after  the  collection  of  the  background  point. 
TEMTADS  backgrounds  were  reviewed  more  thoroughly  than  MetalMapper  backgrounds  based 
on  updates  to  UX-Analyze  from  the  two  surveys  that  added  additional  tools  for  doing  so. 

6.2  PARAMETER  ESTIMATION 

All  data  points  were  inverted  using  UX-Analyze  to  determine  modeled  parameters  for  each  target 
that  included  the  location,  size,  and  orientation  of  the  source  object;  the  polarizability  of  each  axis 
of  the  object;  and  information  regarding  the  quality  of  the  data  and  the  relative  match  between  the 
inverted  data  and  the  expected  model. 

Target  inversion  was  performed  using  the  UX-Analyze  batch  processing  mode  with  both  the  single 
and  multiple  object  solvers.  All  results  for  a  given  target  (single  object  and  all  multiple  object 
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results)  were  compared  to  determine  which  method  returned  a  result  more  indicative  of  TOI.  It 
should  also  be  noted  that  an  update  to  UX- Analyze  changed  the  way  the  multiple  object  inversions 
were  performed.  The  multiple  object  solver  determines  the  number  of  objects  present  using 
algorithms  that  group  point  clouds  into  one  or  more  sources.  Earlier  UX-Analyze  versions  used 
only  one  algorithm  for  this  process  and  returned  one  result.  The  updated  version  of  UX-Analyze 
used  for  MMR,  WMA,  and  the  TEMTADS  data  at  Camp  Beale  used  four  different  grouping 
algorithms  and  returned  four  possible  results.  During  target  classification  (see  Section  6.3.3),  the 
result  that  contained  a  source  with  the  closest  match  to  a  TOI  was  selected  for  use. 

6.3  TRAINING  AND  CLASSIFICATION 

6.3.1  Confidence  Metrics 

The  polarizability  curves  developed  for  each  target  were  compared  to  a  library  of  known  curves 
compiled  using  test  stand  data  and  test  pit  data  from  each  site.  Libraries  were  tailored  specifically 
to  the  TOI  expected  at  each  site.  An  initial  comparison  between  the  measured  targets  and  the 
library  data  was  performed  using  a  weighted  confidence  metric  for  the  three  primary 
polarizabilities  (size:  1,  shape  1:  0.5,  shape  2:  0.5).  For  projects  that  used  the  most  current  version 
of  UX-Analyze,  the  best  multiple  object  solver  result  (see  Section  6.2)  was  selected  using  the 
weighted  metric.  In  addition  to  the  weighted  confidence  metric  generated  during  the  initial 
comparison  of  the  results  to  the  library,  three  more  metrics  were  generated  for  each  target: 

1.  3-curve  metric  -  size:  1,  shape  1:1,  shape  2:  1 

2.  2-curve  metric  -  size:  1,  shape  1:1,  shape  2:  0 

3.  1-curve  metric  -  size:  1,  shape  1:  0,  shape  2:  0 

As  a  first  step,  each  target  was  examined  by  looking  at  a  figure  showing  the  two  closest  matches 
for  the  weighted,  2-curve,  and  1 -curve  comparisons  to  the  library.  Results  were  generally  grouped 
into  one  of  five  categories: 

1.  All  three  polarizability  curves  (pi,  P2,  P3)  were  usable 

2.  Only  two  of  the  curves  (pi,  P2)  were  usable 

3.  Only  pi  was  usable 

4.  No  usable  curves,  but  it  was  determined  unlikely  that  there  was  a  target  large  enough  to  be 
TOI  in  the  acquisition  location 

5.  “Can’t  Analyze”  (no  usable  curves,  and  it  was  considered  likely  that  the  curves  were 
unusable  despite  the  existence  of  a  source  potentially  large  enough  to  be  TOI) 

The  difference  between  targets  deemed  likely  to  have  a  source  potentially  related  to  TOI  (“Can’t 
Analyze”)  and  those  with  unusable  curves  due  to  a  small  or  nonexistent  source  was  determined  based 
on  the  original  EM61  response  for  the  target,  and  the  beta  noise  value  and  the  signal  strength 
calculated  for  that  target  during  inversion.  Additionally,  the  instrument  operator  (MetalMapper)  or 
reacquisition  team  (TEMTADS)  noted  locations  for  which  no  identifiable  signal  was  observed 
during  collection/flagging.  All  results  were  considered  usable  for  these  targets  regardless  of  noise 
values  and  polarization  curve  appearance  at  the  discretion  of  the  data  processor. 
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6.3.2  Training  Data 

Training  data  were  primarily  derived  from  previous  testing  of  the  MetalMapper  using  various  inert 
MEC  items  and  the  test  pit  data  collected  prior  to  the  start  of  data  collection.  These  data  were  used 
to  create  the  libraries  of  polarizabilities  for  standard  munitions  types.  In  addition  to  the  library 
comparison,  the  locations  of  targets  within  parameter  space  plots  generated  using  a  decay  versus 
size  comparison  were  examined.  These  plots  were  be  used  to  identify  groups  of  targets  potentially 
indicative  of  a  munition  that  was  not  expected  at  a  given  site.  Munitions  tend  to  group  together  in 
feature  space,  and  clusters  of  targets  not  already  classified  as  TOI  were  considered  potentially 
indicative  of  unexpected  TOI.  One  or  two  examples  from  each  identified  cluster  were  typically 
requested  as  training  data.  Additionally,  labels  were  also  requested  for  1)  targets  that  matched 
library  items  to  a  particularly  high  degree  according  to  curve  matching,  but  that  exhibited 
decay/size  characteristics  similar  to  those  items  2)  for  targets  plotting  in  the  immediate  vicinity  of 
any  apparent  decay/size  break  line  identified  between  suspected  TOI  and  non-TOI  to  refine  the 
locations  of  these  lines. 

6.3.3  Target  Classification 

Classification  was  primarily  accomplished  using  the  four  confidence  metrics  generated  for  each 
target  during  the  comparison  to  the  library  data,  the  weighted  metric,  the  3-curve  metric,  the 
2-curve  metric,  the  1 -curve  metric,  and  a  feature  space  plot  that  took  into  account  values  calculated 
for  the  geophysical  size  and  decay  constant  (calculated  for  time  gates  8-32  for  the  MetalMapper, 
and  14-63  for  the  TEMTADS)  for  each  target.  A  classifier  was  developed  specifically  for  each 
site  using  thresholds  for  the  four  confidence  metrics  and  for  decay/size  values  based  on  the  space 
plots.  The  classifier  used  for  each  project  is  described  in  detail  in  the  individual  Demonstration 
Reports. 

6.4  DIG  LISTS 

Each  target  on  the  master  dig  list  for  each  site  was  given  a  priority  and  a  confidence  metric  based 
on  the  classifier,  with  priorities  dependent  on  how  the  target  was  classified  (e.g.,  “Can’t  Analyze” 
=  priority  0,  training  =  priority  1,  target  classified  as  TOI  using  all  three  polarizability  curves  = 
priority  2)  and  a  confidence  metric  based  on  the  best  match  to  an  item  in  the  library.  Dig  lists  were 
ordered  by  ascending  priority  and  descending  confidence  metric.  The  dig/no-dig  threshold  for  each 
site  was  set  at  the  end  of  one  priority  and  the  beginning  of  another,  although  the  exact  priority 
numbers  were  site-dependent. 
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7.0  PERFORMANCE  ASSESSMENT 

7.1  EM61-MK2  DATA  COLLECTION  OBJECTIVES/REACQUISITION 
OBJECTIVES 

These  objectives  were  developed  based  on  standard  MMRP  project  DQOs.  While  the  EM61  data 
collection  and  reacquisition  processes  were  important  for  detecting  and  accurately  locating  targets 
for  the  later  advanced  EMI  sensor  surveys,  a  comprehensive  evaluation  of  the  effectiveness  is  not 
considered  necessary  for  this  report.  The  EM61  and  reacquisition  surveys  went  as  expected. 
Further  detail  regarding  individual  performance  objectives  is  contained  in  the  Demonstration 
Reports  for  sites  where  these  surveys  were  performed. 

7.2  CORRECTLY  IDENTIFY  SEED  ITEMS  IN  THE  INSTRUMENT  VERIFICATION 
STRIP 

■  The  performance  objective  of  detecting  >98%  of  IVS  seeds  with  a  confidence  metric  of  >0.7  was 
passed  for  all  sites  with  the  exception  of  the  former  WMA.  Field  results  for  the  classification  of 
IVS  items  at  the  former  WMA  typically  resulted  in  larger  items  than  were  actually  present  (all 
WMA  IVS  items  were  37-mm  projectiles).  It  was  suspected  that  the  geology  at  the  site  was 
contributing  response  to  the  data  collected  over  the  seed  items  despite  the  background  corrections 
performed  for  each  point.  Parsons  continued  data  collection  for  this  site  despite  the  IVS  results, 
while  BTG  performed  the  analysis  and  classification  work,  which  included  more  rigorous  methods 
of  removing  geologic  effects  than  were  performed  by  Parsons. 

7.3  CORRECTLY  POSITION  SENSOR  RELATIVE  TO  THE  SOURCE  / 

CORRECTLY  POSITION  METALMAPPER  RELATIVE  TO  EM61-MK2 
TARGET 

These  performance  objectives  were  not  considered  for  some  earlier  sites  (Camp  Beale 
demonstration  project,  PMTMA,  Fort  Sill).  These  objectives  were  added  as  target  collection  goals 
based  on  the  distance  between  collection  points  and  actual  seed  locations  on  these  projects.  In 
these  earlier  projects,  Institute  for  Defense  Analysis  scoring  did  not  fail  a  TOI  incorrectly  classified 
as  non-TOI  if  the  collected  point  failed  either  of  these  objectives.  For  all  later  projects  except 
WMA,  the  metrics  were  passed  for  100%  of  the  targets,  or  the  target  for  which  one  or  both 
objectives  was  not  achieved  was  identified  as  a  “Can’t  Analyze”  requiring  intrusive  investigation 
on  the  dig  list.  The  number  of  targets  labeled  as  digs  due  to  exceedances  of  these  metrics  over  the 
course  of  the  demonstration  projects  was  negligible  (<1%). 

At  the  former  WMA,  an  initial  modeled  location  for  each  MetalMapper  target  was  determined 
using  an  in-field  QC  program,  and  a  re- shot  was  collected  for  any  target  for  which  the  source 
modeled  farther  than  30  cm  from  the  collected  point.  No  additional  re-shots  were  collected  if  a 
source  did  not  model  underneath  the  first  re- shot  location.  The  field  team  did  not  have  sufficient 
time  to  re- shoot  any  targets  that  did  not  have  sources  within  40  cm  of  the  collected  location  despite 
the  appearance  of  a  modeled  source  within  30  cm  of  the  collection  point  in  the  field  (i.e.,  the  final 
model  did  not  agree  with  the  in-field  model). 
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7.4  MAXIMIZE  CORRECT  CLASSIFICATION  OF  TOI 


Table  7-1  summarizes  Parsons’  demonstration  project  results  with  regard  to  the  detection  of  all 
TOI  present  on  the  respective  sites. 

Table  7-1.  Demonstration  Results  and  TOI 


Demonstration  Project 

%  TOI  Classified  Correctly 

Former  Camp  Beale  Demonstration 

99.2%  ; 

Former  Camp  Beale  Pilot  Study 

100% 

Former  PMTMA 

100% 

Fort  Sill 

97.5% 

MMR 

92.1%  (Phase  I),  83%  (Phase  II)  1 

Former  Camp  George  West 

cued  data  acquisition  and  classification 
performed  by  Sky  Research 

Former  WMA  MetalMapper 

classification  performed  by  BTG 

Both  the  Camp  Beale  Pilot  Study  and  PMTMA  data  sets  passed  the  objective  of  correctly 
classifying  all  TOI  as  dig  targets.  The  misses  at  the  remaining  sites  were  generally  considered 
reasonable:  The  one  Camp  Beale  Demonstration  miss  was  due  to  an  exceedance  of  the  allowable 
cued  collection  to  dynamic  target  offset  (not  yet  an  objective  for  that  site),  all  but  one  of  the  Fort 
Sill  misses  were  extremely  small  (40-mm)  grenade  fragmentation  balls  that  were  not  expected  to 
be  present  without  the  rest  of  the  grenade,  and  the  overall  goal  of  the  MMR  project  was  to  reduce 
the  amount  of  large  ordnance  containing  energetic  filler  present  onsite  rather  than  to  remove  every 
TOI  present.  Almost  all  of  the  missed  TOI  at  MMR  were  thin-walled,  60-mm  illumination  rounds, 
which  produced  a  rapidly  decaying  response  typically  unexpected  for  TOI.  The  MMR  Report 
concluded  that  the  project  was  successful  if  the  illumination  rounds  were  not  a  significant  concern, 
or  if  the  illumination  rounds  could  be  classified  as  TOI  if  significantly  more  excavation  of  non- 
TOI  was  acceptable  to  excavate  the  majority  of  these  items. 

7.5  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOI 

Table  7-2  summarizes  Parsons’  demonstration  project  results  for  the  reduction  in  non-TOI  digs 
achieved  at  each  site: 


Table  7-2.  Demonstration  Results  and  Non-TOI 


Demonstration  Project 

%  Reduction  in  Non-TOI  Digs 

Former  Camp  Beale  Demonstration 

78.2%  ' 

Former  Camp  Beale  Pilot  Study 

90.9%  ; 

Former  PMTMA 

78.5%  : 

Fort  Sill 

61.3%  : 

MMR 

78% 

Former  Camp  George  West 

cued  data  acquisition  and  classification 
performed  by  Sky  Research 

Former  WMA  MetalMapper 

classification  performed  by  BTG 
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All  of  the  Parsons  classification  projects  successfully  classified  >50%  of  the  clutter  on  each  site  as 
clutter. 

7.6  CORRECTLY  IDENTIFY  TYPE  OF  TOI/CORRECTLY  IDENTIFY  TYPE  OF 
NON-TOI 

The  ID  of  the  type  of  TOI  and  non-TOI  objectives  were  only  applicable  to  the  Camp  Beale  Pilot 
Study.  More  than  91%  of  the  TOI  at  Camp  Beale  were  identified  correctly,  and  86%  of  the  non- 
TOI  were  identified  correctly.  Both  results  were  above  their  respective  objectives  of  75%  and  50%. 

7.7  MINIMIZE  NUMBER  OF  “CAN’T  ANALYZE”  TARGETS 

The  Camp  Beale  Demonstration  was  the  only  project  to  have  a  >2%  “Can’t  Analyze”  rate,  and  the 
rate  for  that  project  was  <3%.  This  was  the  first  project  completed,  and  lessons  learned  at  this  site 
allowed  better  comparison  of  EM61  target  data  with  advanced  sensor  parameters.  Application  of 
these  comparisons  resulted  in  a  lower  percentage  of  “Can’t  Analyze”  targets  at  subsequent  sites. 

7.8  CORRECT  ESTIMATION  OF  TARGET  PARAMETERS 

The  standard  deviation  of  the  offsets  between  the  predicted  and  actual  locations  of  the  source 
objects  on  most  of  the  projects  was  slightly  above  the  required  parameters  of  15  cm  horizontal 
distance  and  10  cm  vertical  distance.  However,  these  comparisons  generally  took  all  of  the  targets 
collected  into  account.  Comparison  of  the  offsets  for  only  those  targets  identified  as  digs  on  the 
ranked  dig  lists  and  for  only  those  targets  that  ended  up  being  TOI  were  usually  much  closer  to 
passing  the  metrics  than  the  entire  data  set.  It  appears  that  the  locations  for  the  much  smaller  clutter 
that  account  for  the  non-dig/non-TOI  targets  were  modeled  much  less  accurately  than  the  locations 
for  the  larger  objects  that  typically  ended  up  as  dig  targets  on  the  dig  list.  Sites  with  very  high 
anomaly  densities  and  larger  TOI  (i.e.,  Fort  Sill  and  MMR)  also  tended  to  exhibit  larger  offsets 
than  the  sites  with  lower  target  densities  and  smaller  TOI.  This  appears  to  have  been  a  combination 
of  less-accurate  modeled  locations  due  to  the  increased  density  of  sources  and  offsets  caused  by 
the  predicted  location  modeling  on  one  end  of  a  large,  somewhat  inverted  TOI  while  the  recovery 
location  was  measured  in  the  middle.  This  objective  was  passed  for  both  the  horizontal  and  vertical 
offset  during  the  Camp  Beale  Pilot  Study,  which  was  the  only  project  to  include  the  intrusive 
investigation  of  only  targets  identified  as  digs  on  the  ranked  dig  list. 
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8.0  COST  ASSESSMENT 


Cost  assessments  were  evaluated  by  comparing  the  costs  of  advanced  EMI  sensor  classification 
techniques  incorporated  with  conventional  intrusive  costs  to  traditional  intrusive  costs  alone.  The 
advanced  sensor  costs  included  the  cost  of  instruments,  surveying,  surface  sweep,  seeding, 
dynamic  detection  survey,  cued  data  survey  and  analysis,  and  intrusive  investigation;  the 
conventional  intrusive  costs  included  the  cost  of  instruments,  surveying,  surface  sweep,  dynamic 
detection  survey,  and  intrusive  investigation. 

8.1  COST  MODEL 

The  cost  model  shown  in  Table  8-1  is  an  example  showing  the  current  expected  costs  to  perform 
an  advanced  sensor  cued  survey  on  a  relatively  standard  site.  The  model  is  based  on  a  site  with 
approximately  2,000  targets,  uses  current  expected  production  rates,  and  removes  site-specific 
considerations  that  would  add  significant  costs,  e.g.,  extremely  remote  sites,  either  for  shipping 
purposes  or  for  time  lost  during  transit  from  a  field  office  to  the  data  collection  location  (both 
former  WMA). 


Table  8-1.  Details  of  Costs  Tracked 


Cost  Element 

Data  Tracked  During  Demonstration 

Estimated  Costs 

MetalMapper  Survey  Costs 

MetalMapper  rental  ($500/day;  22  days  including 

$11,000 

shipping  time) 

MetalMapper  prep  fee  (project) 

$1,000 

MetalMapper  shipping  (project) 

$2,800 

Tractor  rental  (project) 

$1,365 

Instrument  costs 

Tractor  mob/demob  (project) 

RTK  GPS  cost  ($800/week;  3  weeks) 

$250 

$2,400 

Shipping  (RTK  GPS,  etc.;  project) 

$324 

Total 

$19,139 

Per  target 

$9.56 

Field-related  labor  (two  geophysicists,  UXO 
Technician  II),  equipment  setup,  test  pit  data 

$42,083 

Survey  Costs 

collection,  cued  data  collection,  preprocessing, 
initial  target  inversion  for  QC  checks,  non¬ 
equipment  direct  costs  (e.g.,  per  diem,  hotel,  fuel) 

Per  target 

$21.04 

All  processing  and  analysis  performed  following  the 

$8,000 

Analysis  Costs 

completion  of  field  activities 

Per  target 

$4.00 

Intrusive  Costs 

All  costs  related  to  the  intrusive  investigation 

$200,000 

Investigations 

Cost  per  anomaly  to  intrusively  investigate 

$100 
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8.2  COST  DRIVERS 

Based  on  the  factors  described  above,  the  total  per  target  cost  for  the  MetalMapper-related  work 
is  $34.60.  The  estimated  production  rate  for  the  hypothetical  project  was  300  targets  collected  per 
day,  with  one  day  of  re-shots,  for  a  total  of  8  days  of  data  collection.  The  estimated  production  rate 
also  included  2  days  of  set-up,  1  day  of  take-down,  and  1  day  of  test  pit  data  collection,  for  a  total 
of  12  days  onsite.  Five  days  of  transport  each  way  for  MetalMapper  rental  was  also  assumed.  The 
intrusive  costs  have  been  determined  based  on  actual  costs  from  numerous  previous  MMRP 
projects. 

8.3  COST  BENEFIT 

For  a  production  removal  action  project  with  10,000  targets  selected  for  investigation,  the 
$34. 60/target  cost  for  data  collection  and  processing  is  used  and  the  intrusive  costs  are  expected 
to  be  closer  to  $100/target.  The  cost  model  assumes  a  75%  reduction  in  the  number  of  clutter  items 
dug  during  the  intrusive  investigation,  which  is  lower  than  most  of  the  demonstration  projects  and 
considerably  lower  than  the  results  seen  on  more  recent  non-demonstration  projects  completed  by 
Parsons.  The  use  of  advanced  sensors  would  yield  a  potential  cost  savings  of  $422,000,  based  on 
the  following  assumptions: 

•  10,000  targets  at  $  100/target  for  intrusive  investigation  =  $1,000,000. 

•  Reduction  of  7,500  targets  =  a  reduction  of  $750,000  in  excavation  costs. 

•  MetalMapper  costs  for  collecting  and  analyzing  10,000  targets  at  $34. 60/target  =  $346,000. 
Total  net  savings  under  this  scenario  equals  $404,000  (approximately  40%). 
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9.0  IMPLEMENTATION  ISSUES 

9.1  IMPLEMENTATION  ISSUES 

The  largest  implementation  issues  on  the  various  projects  were  the  result  of  failures  of 
MetalMapper  transmitters  and  receivers.  Diagnosis  and  repair  of  transmitter  problems  required  2- 
5  days  of  field  time  on  the  Fort  Sill  and  Camp  Beale  Pilot  Study  projects.  Receiver  issues  were 
typically  identified,  determined  to  be  relatively  unimportant  given  the  location(s)  of  the 
malfunctioning  receiver  (always  on  outer  receiver,  non-Z  orientation  wrappings),  and  largely 
ignored  in  the  field.  Responses  from  these  receivers  were  not  used  in  inversion,  and  repairs  were 
completed,  as  necessary,  following  data  collection. 

The  serial-to-USB  cable  used  to  stream  data  from  the  GPS  and  IMU  to  the  MetalMapper  computer 
on  the  Camp  Beale  and  Fort  Sill  projects  also  led  to  numerous  software  crashes  on  those  projects. 
Newer  versions  of  these  cables  fixed  the  problem,  so  this  issue  is  no  longer  considered  a  concern 
moving  forward. 

The  largest  implementation  issue  with  the  TEMTADS  was  a  re-shot  rate  nearing  25%  for  the  Camp 
Beale  Pilot  Study.  The  TEMTADS  survey  area  was  tree  covered  and  much  rockier  than  the 
MetalMapper  survey  area,  and  it  is  suspected  that  many  of  the  TEMTADS  targets  were  selected 
based  on  geophysical  noise  in  the  EM61  data. 

9.2  LESSONS  LEARNED 

The  following  are  lessons  learned  from  the  demonstration  projects: 

1.  The  selection  of  the  vehicle  used  to  transport  the  MetalMapper  has  been  an  issue  at 
various  sites.  In  addition  to  the  tractors  used  for  most  of  the  demonstration  projects, 
Parsons  has  used  extendable  reach  forklifts  and  a  tracked  skid  steer  to  carry  the 
MetalMapper.  Parsons  has  also  used  three  different  sled  configurations  based  simply  on 
what  was  available  for  each  project.  Most  of  the  demonstration  project  sleds  were  the 
original  white,  wooden  version  that  attached  to  the  rear  three-point  hitch  on  a  tractor.  A 
similar  type  of  sled  was  used  for  another  project  at  Camp  Sibert,  although  the  mount  that 
arrived  with  the  sled  only  allowed  for  the  attachment  of  the  sled  to  the  bucket  on  the  front 
of  the  tractor.  It  also  arrived  at  the  site  damaged  to  the  point  that  it  would  not  fit  on  the 
bucket,  and  the  mount  was  eventually  taken  to  a  local  shop  to  be  modified  for 
compatibility  with  the  three-point  hitch  on  the  Sibert  tractor.  A  completely  different  type 
of  sled  (yellow,  composite  version)  that  only  attaches  to  a  skid  steer  or  skid  steer  adapter 
was  used  for  the  Hawthorne  Army  Depot  project,  and  it  ended  up  being  far  too  heavy  for 
the  tractor  rented  to  carry  it.  The  tractor  also  proved  to  be  the  wrong  vehicle  for  the 
Hawthorne  Army  Depot  as  it  quickly  became  apparent  that  it  was  going  to  get  stuck 
repeatedly  in  the  sand  that  characterizes  the  site. 

The  identification  of  the  sled  to  be  used  for  a  given  project,  the  specific  mount  attached 
to  that  sled,  the  correct  vehicle  for  navigating  the  site’s  terrain,  and  an  appropriate  means 
of  attaching  the  sled  to  the  vehicle  should  be  done  long  before  the  field  team  arrives  to 
start  putting  the  equipment  together. 
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2.  The  instruments,  particularly  the  MetalMapper  when  attached  to  the  GPS  and  IMU,  are 
very  complex  pieces  of  machinery  that  do  not  have  readily  available  replacement  parts.  On 
standard  MMRP  projects,  it  is  generally  possible  to  have  spare  parts  onsite  for  geophysical 
instruments  such  as  the  EM61.  Even  if  it  is  not  economically  feasible  to  have  an  entire 
replacement  unit,  spare  cables  are  generally  included  with  these  sensors,  and  replacement 
sensors  can  be  overnighted  from  rental  companies.  This  is  not  the  case  with  the 
MetalMapper.  The  failure  of  a  transmitter  or  receiver  due  to  circuit  board  failure  in  the 
DAQ,  the  sensor  itself,  or  a  broken  cable  between  the  two  can  result  in  days  of  lost  time. 
At  this  point  in  the  development  of  the  instruments,  there  is  little  that  can  be  done  to  prevent 
this  need  aside  from  taking  care  of  the  equipment. 

3.  No  significant  data  transfer  (file  transfer  protocol  [FTP]  site)  or  processing  issues  were 
noted  during  the  project.  The  methods  used  worked  well,  and  Parsons  has  continued  to  use 
the  same  methods  on  numerous  subsequent  projects.  The  time  required  for  data  processing 
is  likely  to  increase  on  future  projects  given  updates  to  UX-Analyze  and  the  ability  to 
generate  additional  summary  data  for  each  point.  Although  still  a  significant  effort,  the 
only  data  products  generated  by  Parsons  for  the  demonstration  projects  were  the  Geosoft 
databases,  figures  showing  the  polarizability  curves  for  each  target,  and  space  plots.  UX- 
Analyze  now  allows  the  creation  of  decision  maps  that  can  include:  the  polarizability 
curves,  the  fit  results,  a  space  plot  showing  the  target  in  question  highlighted  among  all  of 
the  other  targets  in  the  project  or  a  subset  of  those  targets,  and  other  user-selectable  data 
points.  The  USACE  has  requested  decision  maps  on  subsequent  projects.  While  certainly 
useful  for  more  quickly  illustrating  why  a  specific  classification  decision  was  made  for 
each  target,  generating  three  or  four  base  maps  for  inclusion  in  approximately  300  decision 
maps  per  day  is  much  more  time  consuming  than  just  generating  the  polarizability  curves. 
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APPENDIX  A  POINTS  OF  CONTACT 
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Role  in  Project 
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303-764-1927 

Greg  .Van  @  Parsons  .com 
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